Sentiment Analysis with RNNs

Here's where things get a bit more complicated. Consider the following two reviews:

One is positive, and one is negative, even though they have very similar words. In a one-hot encoding, or in a bag of words model, their vectors would be very similar, so one would imagine that the output is very similar too.

One way to fix this is to consider the order of the words. This is where RNNs and LSTMs come to the rescue. With an RNN (LSTM) architecture, we'd be feeding the one-hot-encoded word vectors one by one. At each point, the model takes as input, the previous output, joined with the new word, in order to produce an output. The final output is an encoding of the sentence.

Once we have the encoding of this sentence, we run that through one or more dense layers, which will then get trained to predict the sentiment of the review.

This process is illustrated in the video below.

Review Material

For the next lab, if you need a refresher on either RNNs or Keras, please feel free to check the Extracurricular Material for optional lessons on those topics.

• Recurrent Neural Networks
• Keras